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We consider upper exponential bounds for the probability of the 
event that an absolute deviation of sample mean from mathematical 
expectation p is bigger comparing with some ordered level e. These 
bounds include 2 coefficients {a,/3}. In order to optimize the bound 
we are interested to minimize linear coefficient a and to maximize 
exponential coefficient (3. Generally, the value of linear coefficient a 
may not be smaller than one. The following 2 settings were proved: f ) 
{1,2} in the case of classical discreet problem as it was formulated by 
Bernoulli in the 1 7th century, and 2) {I, ^zp } m the general discreet 
case with arbitrary rational p and e. The second setting represents 
a new structure of the exponential bound which may be extended to 
continuous case. 

1 Introduction 

Let X, Xi,X2, ■ ■ ■ be a sequence of independent and identically distributed 
random variables Pr(X = 1) = p, Pr(X = 0) = 1 — p. 
Ya. Bernoulli proved [14J that 
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P = frjj " = M r + s), e = ^r^, and fc, r, s are arbitrary natural numbers. 
As far as £i > £2 > = ne 2 we can make conclusion that 

^ / 1 / 21 r + l x 1 / 2i s + 1 \\ 
C = mm ( exp (ne log j, exp (ne log )). 

s — 1 r r — 1 s 

Therefore, we can formulate the law of large numbers 

1 n 

lim Pr(|- > Xi-p\ > e) = 0, 

?i— >oo 77, ' 

i=l 

where r and s are assumed to be fixed and k — > 00. 

Markov [TO], [12] considered case of arbitrary n,p and e. Uspensky [16] 
extended results of Markov further and derived the first exponential bound 

1 n 

Pr(|- Vl, -p| > e) < aexp{-/3e 2 n} (1) 
n ^— ' 

i=l 

with coefficients a = 2,/3 = 0.5. Additional and more detailed historical 
notes may be found in the Section "Existing Inequalities" [I]. 

Hoeffding [8] developed methods of [16], [4] and proved generally that 
a = /3 = 2. Note that similar exponential bounds for the empirical distribu- 
tion functions were presented in [5] and [13] . 

In the Section [3] we prove that the following values of the constants a = 1 
and (3 = 2 may be used in the bound (PQ) in the discreet case formulated 
and considered by Ya. Bernoulli. It is demonstrated in the Section \3. II that 
value of a can not be smaller than one. The following Section 13.21 proves 
one-sided inequalities using methods and results of the Theorem [TJ 

Section U] introduces a new structure of the exponential bound in the 
general discreet case with arbitrary rational p and e. The best bound in 
asymptotical sense corresponds to the bigger value of the exponential co- 
efficient. However, as it is discussed in the Section [4. H the value of linear 
coefficient is also very important because of the practical reasons. 

Section [5] represents an extension of the methods and results of the Sec- 
tion [5] to the continuous case. 

Section T5 . 1 1 considers particular application of the bounds to the normal- 
ized sum of random variables. It is demonstrated that all 3 types of bounds 
are asymptotically equivalent. In this way, advantage of the propose bounds 
comparing with Hoeffding's bound is absolute. 
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Note that using results of [2] and [3] we can extend bounds for Bernoulli 
random variables to the case of arbitrary bounded random variables. 

Also, we note paper [9] where similar exponential bounds were con- 
structed for Markov chains. Generally, exponential bounds proved to be 
very effective in order to define required size of the sample in order to en- 
sure proper quality of estimation, see, for example, [TH [T5| [T7]. 



2 Main lemma and definitions 

We will use essentially different approach comparing with [8] and [I]. This 
technique is based on the properties of convex (concave) functions applied 
to the binomial coefficients. The following Lemma formulates the core of 
the methods which are employed in the Sections [31 H] and [5j 

Lemma 1 Suppose that <p(t), 1 < t < n, is a function with continuous 
second derivative where n is any natural number. Then, 

J>(j) > M^-) (2) 

3=1 

if <p'(t) < 0, ip"(t) > 0, 1 < t < n (convex case); 

£ vQ) > n m+m (3) 

3=1 

if (p'(t) > 0,(p"(t) < 0, 1 < t < n (concave case). 
Proof: The following representations are valid 




YJj=o Vfi 1 + j) + - j)] + Vi™ + 1) if n = 2m + l, 



where m is a natural number. We obtain required bounds combining above 
equation with 

n + 1 

i P (l+j)+ip(n-j)>2ip(^-)>2 V >(m + l),l<j<m-l, 
in the convex case, and with 

2ip(m + 1) > <p(l + j) + (f{n - j) > ip(l) + tp(n), 1 < j < m - 1, 
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in the concave case. ■ 

The following notations will be used below 



-i n \ n 

Vt :=Pr(0<-Vl»-p< £ ), Vt :=Pr(e<-y j X i -p) 



n ' - n : 



■y n 1 n 

7>f :=Pr(0<p--^^ <e), V 2 := Pr(e < p - - ^ X ih 



n ' n 

i=i t=i 

where m = kr. 

Assuming that n = k(r + s) we form groups of binomial probabilities of 
the equal size k 

m—l—(j—l)k 



S j: = ( ^ )p € (l-p) n - £ ,J = l,...,r-(left groups), 

e=m-kj ^ ' 

Zj-.= y ™ J A 1 ~ P) n ~'> J = !. • • • . fright groups) 



£=m+l+(j-l)fc 

Then, we consider relations of the corresponding binomial coefficients 
from the neighbor groups 

n \ 

y m-j J ^rn-m + j + v 

A U) ■= ~, r- = II ; = l,...,fc(r - 1 , 

' n \ 1A : m — k — i + v 

v=l 



m — k — j 



(4a) 



n 

/ n \ ±A - n — m — k — i + v 

m + k + j J 

(4b) 

where s, r > 2. 
By definition 

n s r 

Pr (i - E ^ - p\ > £ ) = ^ + +^ = E ^ + E s i- ( 5 ) 

i=l j=2 j=2 

Remark 1 JVoie i/iai T 7 ^ = if s = 1, and T 2 = if r = 1. 
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3 Bernoulli problem 



We exclude from consideration the trivial cases: p = and p = 1, and assume 
that < p < 1. As it will be demonstrated below the task of estimation of 
is easier comparing with estimation of T 7 ^ - if p > 0.5. On the other hand, 
the task of estimation of is easier comparing with estimation of if 
p < 0.5. As far as the problem is symmetrical, we assume that p > 0.5. 

Theorem 1 Suppose that p = ^n;,n = k(r + s),e = and k,r,s are 
arbitrary natural numbers. Then, 

b ■ Zj+i < Zj,j = 1,. . . ,s - 1, 
where b := exp {2e 2 n}. Therefore, 

(b-l)V+<P+. (6) 
Proof: By (|4bj) . -B(j) is an increasing function of j > 1. Therefore, 

Z i \ t> 1 1 ' ,M.^( 1 -P\ k m + -l)k + l + v /s\k 



X 3 :=^>B{{j-l)k + l)( 1 -^-) =1] 

V P / M 



or 



where 



J - J - n — m — jk — 1 + v 

v=l 

k 

logAj > A; log - + ^<p(j,u), j = 1,. . . ,s - 1, 

, . \ , m + (j — l)k + 1 + v 
<p{j, v) = log ■ 



n — m — jk — 1 + v 
is a convex function of v because r > s or m > n — m, and according to ([2]) 

> / s(m + jk- 0.5k + 1.5) x '' 
3 ~ \r(n — m — jk + 0.5k — 0.5) 

k 



- [r(n-m- jk + 0.5k)) ^ + 1 ' ! 



s{m + jk - 0.5k + 2) 
- jfc + 0.5A;) 

n(j - 0.5) + 2s 
kr(s — j + 0.5) 

Using general inequality 



where 



log (1 + 5) > V5>0, (8) 
2 + o 
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we obtain 

2k(n(j - 0.5) + 2s) 



log A,- > 



' 3 ~ 2(s - j + 0.5)kr + n(j - 0.5) + 2s 
2kn(j-0.5) 

> — i' = l s — 1 (9) 

~ 2srk + {j -Q.b){n-2krY J ^ W 

Furthermore, based on the following properties 
2srk + (j - 0.5) (n - 2rk) = k [r(2s -j + 0.5) + s{j - 0.5)] > 0, 2kr > n, 
we have 



log \j > 



n(j - 0.5) 



sr 

Then, using relations r = |,s = — ^£ we transform above inequality to the 
required form 

ne 2 (2i — 1) 

Xj >exp{ 2p > >exp{2ng 2 (2j-l)},j = l,...,s-l. (10) 



Above equation (|10p completes proof of the Theorem. ■ 

Theorem 2 Suppose that p = ^q^,n = k(r + s),e = ^jpj, and k,r,s are 
arbitrary natural numbers. Then, 

b ■ Sj+i < Sj, j = 1,... ,r- 1. 

Therefore, 

( b -l)p-<p-. (11) 

Proof: By ()4ap . the coefficient A(j) is an increasing function of j > 1. 
Therefore, 

qj --=^> m - l)k + 1) (j^-) (12) 

or 

k 

r 

:.■(') 



log qj > k log - + ^2 ^(i' ' 



where ^0', «) = log m *- jk -i+v +V ■ 
The following condition 



k(r - s - 1) > 2 
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give us a guarantee that the function tp(l,v) is concave. 

Suppose that r = s + 2 and fc>2orr>s + 3 and k > 1. Then, by ([3]) 
we have 

r , r k (n — m + 2)(n — m + A; + 1) 
log > log gi > Hog - + - log (m _ fc)(m _ 1} 

k r(s + l) k / r + J \ fc(r + ,) (13) 

~2 s(r-l) 2 S V s(r-l)/ ~ 2s(r-l)+r + s V ; 

ne 2 9 
> 2ne 2 



2p(l - p) + e(2p - 1) 

if 4p(l - p) + 2e(2p - 1) < 1 or e < ^g^g = p - 0.5. Thus, e = ^ < 
-4- - 0.5 or r > s + 2. 

The case r = s + 2,£; = lis easy to consider: 

( s + 2 ) 2 , „(^ , 3s + 4 A ^ 6s + 8 ^ 7 9 



log 91 = log T^TT) = log I 1 + T^TT) ) ~ 2, 2 + 5s + 4 - 2(7TT) = 7n£ 

or 

(12 - 2 7 )s 2 + (28 - 57)s + 16 - 4 7 > 0. 

Above inequality is valid Vs > 1 if 7 < 4. 

Now, we consider remaining case r = s + 1 (the case r = s was considered 
already in the first part of the proof because the problem is symmetrical). 
Then, ifi(j,v) is convex as a function of v, and, by ([2]), 

[s + l)(ks + jk + 1.5 - 0.5k)\ k fc 



"H .(». -j* -0.5; 1.5*) J =(1+a) ' <14) 

where 

_ 2jsfc + 2s - ks + jk + 1.5 - 0.5A; 
s(ks- jk- 0.5 + 1. 5k) 

Using inequality (jHJ), we obtain 

2k(2jsk + 2s -sk + jk + 1.5 -0.5k) jk 2 
l0g * ^ 2 S 2 fc + g + 2^ + J fc + 1.5-0.5fc - 2JTT = (15) 

or 

(8jfc + 8 -4k- 2kj)s 2 + (8jfc - 4fc + 10 - (2k + l) 7 )s + 2j& + 3 - k 

-(ifc + 1.5-0.5% > 0. 
Above inequality is valid for 7 < 2 assuming that j, s > 1. I 
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Corollary 1 The following upper bound is valid 

1 n 

Pr(|-VXi -p\ > e) < exp{-2e 2 n}, (16) 
n ' 

i=l 

where p = ^r^,n = k(r + s),e = ^rj, and k,r,s are arbitrary natural 
numbers. 

Proof: It follows from the Theorems [1] and [2] that 

j=2 j=2 

The following representation is valid 

d n 1 v i ^ \ ^i=2 S i + Sj=2 Z 3 

Pr(l »S X, - ! " >E) = ^E^TE^' 

It easy to see that f{x) = is an increasing function of x, where C 
is a positive constant. Therefore, 

n i=l + z l + FT + FT 



3.1 Optimal value of the linear coefficient 

Clearly, value of the linear coefficient a = 2 in the exponential bound may 
not be regarded as an optimal. The following Proposition [TJ demonstrates 
that a > 1. 

Proposition 1 Suppose that < p < 1. Then, we can construct sequence 
of deviations e(n) — > so that e 2 n — > and Vo — ^ 1- 

n— >oo n— »oo ~ n— >oo 

Proof: By definition "P = f ^ j //"(I — p)''~'". The following relation is 
valid 

n \ V^" m + j 



(\ 11 — in 

:)~x> 



s 
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Figure 1: a) f(p) = ^(l+p) 1 p , < p < 1; c) function /j,(n) which is defined 
in ([33]k bounds (p?Tj) and (f32|) as a function of e where b) n = 20, d) n = 100. 



By Lemma [TJ 



n \ s , 1 + n + m „ , N , n + m , N , 1+p 



log ( I < (n-m) log 

m / 1 + n - m 



< (n—m) log 



n — m 



(n—m) log 



1-p 



Respectively, 

7>° < (^(l + p) 1 ^)", 
where < pP(l + p) l ~ p < 1 \/p : < p < 1. Figure [T^a) illustrates graph of 
the function /(p) = +p) 1_p ,0 < p < 1. 

Suppose that e = Then, and will be empty sets, and + 



e 2 n 



1 as far as V° — > 0. At the same time, by definition, 



0. 



Corollary 2 Coefficient a in the bound may not be smaller than 1 for 
the arbitrary n > 2 and e > 0. 
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3.2 One-sided inequalities 

In this Section we will use again the property that left groups of binomial 
probabilities may be estimated more effectively if p < 0.5. Analogously, right 
groups of binomial probabilities may be estimated more effectively if p > 0.5. 

Proposition 2 Suppose that p = ^r^,n = k(r + s),<5 = ve = where 
k,r,s and v are arbitrary natural numbers. Then, 

Pr - J]X i -p>6 < -exp 
j/r>s, fp>0.5; ; 



y " 2 " v 2p(l-p)' 



1 n 1 

Pr(p- -Vx* ><*)<- exp (- 



n5 2 



n ^ 2 1 v 2p(l -p)' 



if r < s, (p < 0.5). 

Proof: Combining the equality 



£(2*-1) = i/ 2 -(j-1) 2 ,;/ = 1,...,i/, 

i=j 

and (fTUj) we derive that 

Z,- > Z v+1 exp { 2 p(l-p) },J = 1 >--> u > 

and 

r ne 2 (i/ 2 - (7 — l) 2 ) . . , 

^ *i > Vl£°P{ 2 p(1 _ p) J }' ( 18 ) 

Next, we use the same bound (jlOj) in the opposite direction 

r ne 2 (2i/ + 1). 
Zj >Z i+ iexp{ 2p l _ , },j = i/ + l,...,s-l ; 



and 

V* 7 <r 7 A / ^ 2 (2^ + l) ^' 1 



(19) 
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Again, we make an assumption r > s or p > 0.5 without loss of generality, 
and construct upper bound for the conditional probability 



1 - 1 - F 



j=u+l Zj 



8=1 8=1 £-i3-\ J 

If , ne 2 {2u + l)\^ ,ne 2 (v 2 - (j - l) 2 ) x N 

(20) 

where the last inequality may be regarded as a consequence of (I18p and (I19p . 

Then, we simplify ()20p using properties z^ 2 — (j* — l) 2 — 2i/ — 1 < v 2 —j 2 ,j = 
l,...,u, 

1 - 1 - nft 2 

Pr(- £ A* - p ><5) | - £ A, > p) < exp {- — -}. (21) 

n n f-^ 2p(l — p) 

As a next step we prove that 



Pr( 



j n 1 n 

- ^ Ai < p | p > 0.5) > Pr(- V Ai > p | p > 0.5). 



n ^ — ' n 

i=l i=l 



Let us consider relations of the binomial probabilities which are sym- 
metrical against central point m = kr 

n 

m — i + 1 J fl—p\ 2%l -rj (m - i + 1 + j)(n — m) 
n \ \P/ r~ (n — m — i + j)m 



, i \ 2i-l 
m + 1 



m + i 

>( : — I > l,i = 1, . . . , n — m, 

m 

where we used ([2]). Therefore, 

n 1 n 

Pr(- Vlj>p) = Pr(- > Xi > p I p > 0.5) < 0.5. (22) 
n z — ' n z — ' 

i=l i=l 

By the way, using identical method, one can prove 
1 " 

Pr(- > A; < p I p < 0.5) < 0.5. 

71 < ^ 



n . 

i=i 
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The final upper bound follows from ([2Tj) and ([22]) applied to the Bayesian 
formulae 

Pr(iVl t -p>i) 
n ^-^ 

1 n 1 n i n 

= Pr(-Y / X i >p)-Pr(-Tx t -p>5) \-Tx i>P ). 

i=l 8=1 8=1 

■ 

4 General discreet case 

In this section we assume that p and e may be represented by positive ratio- 
nal numbers with denominator as a number of observations in the sample. 
Respectively, we will cover all possible empirical values of the sample mean 
as an estimator of the probability p. Note that the role of the parameter k 
will be different here comparing with previous section. 

Theorem 3 Suppose that p = — > 0.5, e = and k,m,n are arbitrary 
natural numbers with condition k,m < n. Then, 

b- Z j+ i < Zj,j = 1,... ,s- 1, 

where b := exp {2e 2 n}. Therefore, 

(b-l)V+<V+. (23) 

Proof: The following upper bound may be obtained similarly to ([7]) 

(n — m)(m + 0.5k + 2) , 

logA^fclog^ ^ ' J = l,...,s-l, 24 

m(n — m — U.ok) 

where 2m > n and p + | < 1 (else = 0) . 
It follows from ( [Ml ) that 



logAj > felog(l + 5) (25) 



where 5 



0.5kn+2n-2m 



m(n—m—0.5k) ' 

By (jSJ), we obtain 



fcffcn + 4n — 4m) 7& 2 9 

^ — ; ; — > J — = 7£ n 

2nm — 2m/ — sm + 0.5«:n + 2n — 2m n 
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or 



k + 4-4p>-fk 2p(l -p) + -(1 - 2p) + 2je(l-p). 



Above inequality is valid VA; > 1 and Vp>0.5if7<2. ■ 
The following Theorem 0] introduces a new structure of the exponential 
bound as a main result of this paper. 

Theorem 4 Suppose that p = ~ > 0.5, e = ~, and k,m,n are arbitrary 
natural numbers with conditions 2 < k < n, m < n. Then, 



exp{ 



2e 2 n 



} • S j+1 < Sj,j = 1, . . . ,r - 1. 



Therefore, 



(exp{ 



2e 2 n 



} - 1)P 2 " < Pf 



T + e 2 - 

('i/ie case k = 1 is covered by the Theorem^ with stronger result). 
Proof: Similar to (|13p and assuming that 2m > n + + 2 we derive 
, , , m k (n — m + 2)(n — m + + 1) 

log > fc log + - log — , 

n — m 2 [m — k){m — 1) 



(26) 



(27) 



where j = 1, . . . , r — 1, m > k > 1 (else P2 = 0)- 
The following relations are valid 



(n — m + 2)(n — m + /c + 1) 
(m — A;)(m — 1) 

where 6 = , "?+ fcn 

(m— k)(n— m) 



m 



n — m 



^ m(n — m + A; + 1) 
~~ (m — k)(n — m) 



5, 



By 



we obtain 
log qj > 



k{m + fen) 



> 



or 



or 



2m(n — m + k) + m — kn 
n(m + kn) > 7/c [2m(n — m + k) + m — kn] 



7ne 



2p(l -p + e) + ~- e =jk [2p(l - p) + e(2p - 1)1 + -yep. 
n 



p + k > jk 
Above inequality take place if 



7 



< 



l + e 



2 • 



(28) 
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Let us consider remaining case n < 2m < n + k + 2. Similar to ([7|) we 
have 



m(n — m + 1.5 + 0.5A;) \ . / . 



where 



Qj > [ f V w ' ' 'J = (! + *)*, (29) 
1 (n — m)[m — 0.5/c — 0.5) / 

m + 0.5/cn + 0.5n 



nm — m 2 + 0.5(km + m — kn — n) 

By ([8]), we obtain 

2k(m + 0.5fcn + 0.5n) 7/c 2 2 

> = je n 



2nm — 2m 2 + km + 2m — 0.5n(k + 1) n 



or 



k + 2p + l> 7fc(2p(l - p) + 0.5e(2p - 1)) + je(2p - 0.5). 
Above inequality take place if 

7 £ J^J- (30) 
Note that condition ()30f) is less restrictive comparing with 
Corollary 3 T/ie following upper bound is valid 



n 

I 

Pr 



-> Xi-p > e) < exp{-— — }, 31 
n, ^— ' 1 + e z 



n . 



where p = ~,e = £ and k,m,n are arbitrary natural numbers with condi- 
tions 2 < k < n, n < 2m < In. 

Proof is similar to the Corollary Q] : we obtain required value of the 
parameter /3 as min {2, j^z, ^ e -i } where first value follows from Theorem[3l 
second and third values follow from (|28p and (I30p . 

4.1 Relations between bounds 

The bound ([3T|) improves Hoeffding's bound 



n . 



1 n 

Pr(|-^Xi -p| > e) < 2exp{-2e 2 n} (32) 

if 



0«><M'W = | + J*(l + $), (33) 
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Table 1: The following parameters were used in this example: n = 33, m = 
15, k = 1, . . . , 14, e = -, second column represents real probability of abso- 
lute deviation of the sample mean from p = ^ , next two columns represent 
corresponding bound ([5T]) and Hoeffding's bound (f3"2"j) . 



e True probability Bound §T$i Hoeffding's Bound <JH3> 



0. 


.0606 





.600713 





.78542 


1.569446 


0. 


.0909 





.382439 





.582175 


1.159157 


0. 


.1212 





.220522 





.38456 


0.758396 


0. 


.1515 





.114271 





.227376 


0.43955 


0. 


.1818 





.052796 





.120996 


0.225672 


0. 


.2121 





.021571 





.058319 


0.102638 


0. 


.2424 





.007724 





.025643 


0.041352 


0. 


.2727 





.0024 





.010366 


0.014758 


0. 


.3030 





.00064 





.003884 


0.004666 


0. 


.3333 





.000145 





.00136 


0.001307 


0. 


.3636 





.000027 





.000449 


0.000324 


0. 


.3939 





.000004 





.000141 


0.000071 


0. 


.4242 





.000001 





.000042 


0.000014 


0. 


.4545 










.000012 


0.000002 



Therefore, 



lim n^(n 



log 2 



n— >oo \ 2 

Figure QJc) illustrates behavior of the function \i. 

Remark 2 It follows from \33\) that the number of observations must be big 
enough in order to ensure advantage of the Hoeffding's bound for the fixed 
deviation parameter e: 

log 2 
n > — 7-. 

2e 4 

As a direct consequence, the bound \°J1\) will be so small: 

2e 2 n log 2 1 

exp{ -TT^ }<exp{ -?(iT^) } 

that any further improvement may not be regarded as a significant. Fig- 
ures{J^b)(d) illustrate above fact with relatively small numbers of observa- 
tions n = 20 and n = 100. 
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5 Continuous Case 



Suppose that p and e are arbitrary numbers: 

< p< 1; (34) 

— < e < min {p, 1 — p}. (35) 
n 



Remark 3 The special case e < ^ may be considered easily. In the case 
e > p or e > 1 — p we will have simplified cases because we will need to 
approximate only one probability of deviation or ■ 

We define a central point np which is not necessarily integer. We denote 
by 1) h(n) - number of integer numbers in the left group [np—ne, np); 2) g{n) 
- number of integer numbers in the right group [np, np + ne}. All remaining 
left groups will have h integers with only one possible exception as a last 
group. Symmetrically, all remaining right groups will have g integers with 
only one possible exception last group. 

Let us denote by m the smallest integer in [np, np + ne] . 

According to the construction 

0<p-p<-; (36) 
n 

\e l -e\<-; (37) 
n 

9 :=max{ — ,— } < , 38 

E\ £2 ne — I 

where p := ^,e x := ^-,e 2 := 

Theorem 5 Suppose that conditions p^P and i f^3)) are valid. Then, 

1 -A 2e 2 n 
Pr(|- Y j X i -p\ >e) <exp{e • ip(n,e) - —^}, (39) 

where 

ne(l+e 



H+e(2 + 6e + T^ ) 

^) : = (1 + t . )(1 + e . + i(1 + 3t + ;fe ))- (40) 
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Proof: Again, we make an assumption p > 0.5. Similar to (|24|) we obtain 

ix^i (1 — p)(m + 0.5# + 1.5) , 

logAi >glog v — - - / = fflog(l + <5), (41) 

p(n — m — O.og — 0.5) 

where 5 = m+L5+0 - 5g ~ p ~ pra 
wneie a pn-mp-0.5p-0.5gp - 

By (jHJ), we obtain 

2(m + 1.5 + 0.5g — p — pn) ^ 75 



pn — 2mp — gp — 2p + m + 1.5 + 0.5<? ~~ n 
or 

3 — 2p / 2p 3 

£2 + 2(p - p) H > 7^2 P - 2pp - f?2P 1" P + 0.562 + TT - 

n \ n 2n 



Assuming that 7 < 2 we simplify above inequality according to (|36|) by 
making it stronger 

e 2 + > 7 e 2 (2p(l -p) + -(1.5 - 2p)). 

n n 

As a next step we can split above inequality into 2 inequalities 

, 3 — 2p yin , 

£2 > 2 7 £2P 1 ~P ; > — 1.5 - 2p , 

n n 

which are valid if 

7 < 2. (42) 

Now, we consider approximation in the left direction according to the 
previous framework (|27p 

h , 1 / p \ 2 (n — m + 2)(n — m+l + /i)\ 

log gi > - log — 

2 \\l—pj [m — h){m — 1) I 

h / p(n — m + 1 + h)\ h 

£ 2'°H (!-»>)(*-*) ' H'° E(1+ ^ 

where <5 = ^'"Tt? . 

m—mp—h+tvp 



By ([8]), we obtain 



pn + p — m + /i 7/1 

> — = ye\ 



m — 2mp — h + 2/ip + pn + p n 

or 



p 

£l+p-p+ - > 7£i 

n 



p(l - 2p)+p-£i(l - 2p) + - 

n 
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Table 2: Values of the function exp {s(p(n, e)}. 



n\e 


0.02 


0.05 


0.1 


0.2 


0.3 


0.35 


100 

1,000 

100,000 


1.0412 
1.0221 
1.0211 


1.0697 
1.0582 
1.0572 


1.1436 
1.1336 
1.1326 


1.3737 
1.3652 
1.3643 


1.7612 
1.7566 
1.7561 


2.0357 
2.0349 
2.0348 



Again, we simplify above inequality according to (j36l) and ([371 



1 

In 



2p(l -p) + e{2p - 1) + 



3p- 1 



Therefore, 



7< 



^^ + I (1 + 3e + |. ) - 
The last remaining case corresponds directly to (129j) . 

p(n — m + 1.5 + 0.5fo) 



(43) 



(44) 



log > h log 



(1 -p)(m- 0.5/i -0.5) 



/ilog (1 + 5) 



where S 



pn+p—m+0.5(h-\-l) 
m—pm+0.5(hp+p—h—l) ' 

By ^j, we obtain 



. 2p + 1 
ei + 2(p — p) H > 7£i 



£i . . 2n — 0.5 

p(l-2p)+p + ^(2p-l) + 



We simplify above inequality according to (|36p and ([3 



. e(2p-l) 3p 1 
2 n n 



(45) 



which is valid for any 

< 7 < 2 



4 + £^ 1 / 3£ 9 
— — + - 1 + — + — 

4 n 2 4ra 



(46) 



Finally, we derive required asymptotical relation as a consequence of the 
conditions (|42p . (|44p and ()46p where condition (|44p is the most restrictive. 
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5.1 Asymptotical Bounds for Normalized Sum of Bernoulli 
Random Variables 



Let us denote by F n distribution function of the normalized sum of Bernoulli 
random variables 

,„ _ (47) 
y/np{l-p) 

According to the Central Limit Theorem, 

lim F n (t) = $(*), 

n— loo 

where is a standard normal distribution function. Respectively, it ap- 
pears to be reasonable to use random variable (|47p as a test case in order to 
compare different bounds. 

Proposition 3 The bounds U6\), A31\) and t39\), as an upper bounds for the 
following probability Pr(\r/ n \ > t), are equivalent asymptotically and equal to 

aexp{-2t 2 p(l-p)}, (48) 

where a = 1, and < t,p < 1. 



Proof: We have 

/ n 

PrflTfcl >t) = Pr |- VXi-p| >t 

where t n = t\J p ^ 1 ~ p \ 

Then, we insert t n into (|16p . (|3ip or (j39j) . and take lim if n — > oo. As a 
result, we obtain required formulae (f48l) . ■ 

Remark 4 Using Hoeff ding's bound we will obtain the same asymptotical 
structure but with a = 2. 



6 Concluding remarks 

The Proposition [1] proves that value of the linear coefficient a = 1 can not be 
improved. Taking this fact as a base point we established a new structure 
of the exponential bound (J3TI) . Figures QJa),(c), (d) and Table [U demon- 
strate advantages of the bound (|3ip against Hoeffding's bound if value of 
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e is small enough. The above Section 15.11 demonstrates additional argu- 
ments in support of the proposed bounds, and these arguments cover not to 
only Bernoulli random variables. The area of applications of the proposed 
methods may be extended further, for example, we can consider arbitrary 
bounded random variables or uniform metric for the empirical distributions. 

The paper represents a fresh look at the ideas and methods which Ja. 
Bernoulli proposed in the 17th century. As it was demonstrated in the 
Section [3] the probability of deviation in the classical Bernoulli case may be 
bounded using standard structure of the exponential bound with optimal 
linear coefficient a = 1. It was the first step of this research which was 
completed in 1987 shortly after the 1st World Congress of the Bernoulli 
Society where the author purchased book |14j . 
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